The internet has become a ubiquitous source of information for consumers. The information (content) that can be accessed through the internet includes text, pictures, animation, video and sound recording, and may include software. Users typically access content through the internet by using web browsers. The web browser is a computer program that accesses a web server on the Internet and requests the content. The content is usually transmitted as a web document, which is a text file that contains text, and a set of HTML tags that provide instructions to the browser about how to display the content associated with the web document on the user's screen. The web browser interprets the HTML tags within the web document in order to display the page on the user's screen and determine where to access additional contents from the server. For example, the tags may instruct the browser to display text, change the font size or color of the text, or arrange the content in rows or columns, or access an image file stored at a designated location accessible to the server.
As all users of the internet know, there is a delay between the time the user accesses the web site and the time when the page is fully displayed on the screen. This delay may increase depending on the web site resources and the complexity of the web page. Many complex web pages include content generated by multiple content sources and servers, each optimized for a different kind of web object: databases, streaming media, banner ads, CGI scripts. Those content sources and servers may operate at different speeds or under conditions that may impact the performance of the web browser in interpreting the web document, loading the content or displaying the content. If the generation of content from any content source or server is delayed (or fails altogether), the user of the client browser would perceive either a slow-loading page or an empty spot where the content failed to load. Speed of loading a web page is a critical component of a satisfactory user experience. It is often said that a good web site is a fast web site.
The types of computers and connection speeds of user systems vary widely. For example, a user system that incorporates a 56 k modem (56,000 bits per second) may download data at a rate of approximately 4,000-7,000 bytes per second, while a user system connected though an ISDN may download data at a rate of 64,000 to 128,000 bits per second. At the upper range may be a T1 line that provides an effective bandwidth of 1.5 million bits per second. User systems that are locally connected through Ethernet may communicate at up to 400,000 bytes per second. These varying access bandwidths translate into varying wait time for rendering of web pages. For example, a web page that is primarily text, having a size of approximately 1,000 bytes would render in less than 1 second even with a 56K modem. However, a typical image file or a small sound file having 100,000 bytes may take 20 seconds to download with a 56K modem. A typical video or large sound file having 1,000,000 bytes may take over 3 minutes to download using a 56K modem. The rule of thumb used by web designers is that a web page should download in 10 seconds or less, over a 56K modem. This has been translated roughly to a page size of 70K.
One way that the performance of content generation can be improved is web caching. Web caching is the temporary storage of web objects (such as HTML documents) for later retrieval. The Internet connection of a user's computer is typically its slowest link. Web browsers (e.g. Internet Explorer, Netscape, etc.) use the hard disk of the user's computer to store HTML pages, putting them into a special folder on the hard disk. The first time a user asks for an HTML page, the server transmits a web document to the web browser. The web browser renders the document (i.e. the web browser executes the instructions in the web document, including displaying content in the document, requesting access to remote content on the server, loading the remote content and displaying the remote content). The web browser may also store a copy of the web document and associated content (including remote content) in the user's hard disk. The next time the user requests access to that document, the web browser checks if the date of the file associated with the document on the server is newer than the date of the file containing a copy of the web document and associated content cached in the hard disk. If the date is the same, the web browser uses the web document on the user's hard disk instead of accessing the document on the server. This action reduces the time to render the content on the user's interface (monitor). There are three significant advantages to web caching: reduced bandwidth consumption (fewer requests and responses that need to go over the network), reduced server load (fewer requests for a server to handle), and reduced latency (since responses for cached requests are available immediately). Together, they make the web less expensive and better performing.
The existing uses of caching are used to store a copy of an entire web page after the entire web page has been downloaded. The existing methods of caching provide an advantage only when the web document is accessed for the second time. Such use of caching does not solve the problem of a negative user experience when downloading slow loading content for the first time or loading content from a plurality of sources if some of the sources are slow or non responding.